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Abstract. People usually regard algorithms as more abstract 
than the programs that implement them. The natural way to for- 
malize this idea is that algorithms are equivalence classes of pro- 
grams with respect to a suitable equivalence relation. We argue 
that no such equivalence relation exists. 



1. Introduction 

At the end of his contribution to a panel discussion on logic for the 
twenty-first century [51 page 175], Richard Shore posed, as the last of 
three "probably pie-in-the-sky" problems, the following: 

Find, and argue conclusively for, a formal definition of 
algorithm and the appropriate analog of the Church- 
Turing thesis. Here we want to capture the intuitive 
notion that, for example, two particular programs in 
perhaps different languages express the same algorithm, 
while other ones that compute the same function repre- 
sent different algorithms for the function. Thus we want 
a definition that will up to some precise equivalence re- 
lation capture the notion that two algorithms are the 
same as opposed to just computing the same function. 

The purpose of this paper is to support Shore's "pie-in-the-sky" as- 
sessment of this problem by arguing that there is no satisfactory def- 
inition of the sort that he wanted. That is, one cannot give a precise 
equivalence relation capturing the intuitive notion of "the same algo- 
rithm." We describe several difficulties standing in the way of any 
attempt to produce such a definition, and we give examples indicating 
that the intuitive notion is not sufficiently well-defined to be captured 
by a precise equivalence relation. 
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Following the terminology in Shore's statement of the problem, we 
use "program" for an element of the set on which the desired equiva- 
lence relation is to be defined and we use "algorithm" for an equivalence 
class of programs (or for the abstract entity captured by an equivalence 
class) with respect to an equivalence relation of the sort requested by 
Shore. 

Shore's question has been addressed in a limited context by Yanof- 
sky [2T|. He considers programs only in the form of (fairly) standard 
constructions of primitive recursive functions, and he defines an equiv- 
alence relation on these programs by listing a number of situations in 
which two programs should be considered equivalent. He does not, how- 
ever, claim that the equivalence relation generated by these situations 
completely captures the intuitive notion of equivalence of programs. In 
fact, he explicitly says "it is doubtful that we are complete," and he 
anticipates one of our objections to the "equivalence class of programs" 
view of algorithms by saying that "whether or not two programs are 
the same ... is really a subjective decision." By considering only a lim- 
ited class of programs and limited sorts of equivalence between them, 
Yanofsky obtains a notion of algorithm with pleasant properties from 
the point of view of category theory, but not what Shore asked for. 

We shall also consider only a limited class of algorithms, namely 
deterministic, small-step, non-interactive algorithms. But for our pur- 
poses, such a limitation strengthens our argument. If no suitable equiv- 
alence relation can be found in this limited case, then the situation is 
all the more hopeless when more algorithms (distributed ones, inter- 
active ones, etc.) enter the picture. Any suitable equivalence relation 
on a large class of programs would restrict to a suitable equivalence 
relation on any subclass. 

Small-step algorithms are those that proceed in a discrete sequence 
of steps, performing only a bounded amount of work per step. (The 
bound depends on the algorithm but not on the input or on the step in 
the computation.) Non- interactive algorithms^ of this sort were char- 
acterized axiomatically in They include traditional models of 
computation such as Turing machines and register machines. Most 
people, when they hear the word "algorithm", think of deterministic, 
small-step, non-interactive algorithms unless a wider meaning is explic- 
itly suggested. And until the 1960's, essentially all algorithms in the 
literature were of this sort. Because Shore posed his question in the 
context of traditional recursion theory, it is reasonable to suppose that 



: It is argued in [111 Section 9] that non-determinism is a special case of interac- 
tion. Thus, "non-interactive" implicitly entails "deterministic." 
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he had deterministic, small-step, non-interactive algorithms in mind. 
If so, then it is not only reasonable but in a sense necessary to argue 
our case in the limited context of such algorithms. For if an argument 
against the existence of the desired equivalence relation depended on 
other sorts of algorithms (for example, interactive ones or massively 
parallel ones), then that argument might be regarded as missing the 
point of Shore's question. 

In Sections 2 and 3, we discuss preliminary issues concerning the 
choice of programming languages and their semantics. In Section 4, we 
describe situations where people can reasonably disagree as to whether 
two programs implement the same algorithm. In Section 5, we discuss 
how the intended use of algorithms can affect whether they should 
count as the same. Section 6 contains examples where two quite differ- 
ent algorithms are connected by a sequence of small modifications, so 
small that one cannot easily say where the difference arises. Finally, in 
Section 7, we mention several other domains in which analogous issues 
arise. 



2. Programs 

In order to even hope to capture the notion of algorithm by a precise 
equivalence relation on the set of programs, it is necessary first to have 
a precise set of programs to work with. The present section addresses 
this preliminary issue. We argue that there is a precise set of programs, 
at least for small-step, non-interactive algorithms, but that this fact is 
not as obvious as one might think at first. 

The mention, in Shore's question, of "programs in perhaps different 
languages" suggests that the set of programs should be rather wide, 
encompassing programs written in a variety of languages. But caution 
is needed here, in order to keep the set of programs well-defined. We 
cannot allow arbitrary programming languages, because new ones are 
being created (and perhaps old ones are falling out of use) all the time. 
Even languages that persist over time tend to gradually acquire new 
dialects. We should work instead with a specific set of stable languages. 

Not only should the languages be stable, so as not to present a mov- 
ing target for an equivalence relation, but they should have a precise, 
unambiguous semantics. For example, in some languages, such as C, 
the order of evaluation of subexpressions in an expression may be left 
to the discretion of the compiler, and whether this ambiguity matters 
in a particular program is, in general, undecidable. If deterministic 
algorithms are to be captured by equivalence classes of programs, then 
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ambiguity in the programs is unacceptable. This consideration ex- 
cludes a lot of languages. And pseudo-code, despite being widely used 
to describe algorithms, is even worse from the point of view of precision. 

For many purposes in theoretical computer science, especially in 
complexity theory, it is traditional to use, as the official framework, 
low-level models of computation, such as Turing machines or regis- 
ter machines, that have an unambiguous semantics. Such models of 
computation are, however, inadequate to serve as the domain of an 
equivalence relation capturing the general notion of algorithm. The 
problem is that they can express algorithms only at a low level of ab- 
straction, with many implementation details that are irrelevant to the 
algorithm. 

At first sight, it seems that this difficulty can be circumvented by 
a suitable choice of equivalence relation on, say, Turing machine pro- 
grams. Programs that differ only in the irrelevant details of imple- 
mentation should be declared equivalent. There remains, however, the 
problem that, by including such details, a Turing machine program 
may make the intended algorithm unrecognizable. It is not in gen- 
eral easy to reverse engineer a Turing machine program and figure out 
what algorithm it implements. In fact, it may be impossible; here is 
an example illustrating the problem (as well as some other issues to be 
discussed later). 

Example 1 . The Euclidean algorithm for computing the greatest com- 
mon divisor (g.c.d.) of two positive integers is described by the follow- 
ing abstract state machine (ASM), but readers unfamiliar with ASM 
notation should be able to understand it as if it were pseudo-code. In 
accordance with the usual conventions [10] for ASMs, the program is 
to be executed repeatedly until a step leaves the state unchanged. We 
assume for simplicity that the numbers whose g.c.d. is sought are the 
values, in the initial state, of dynamic miliary function symbols x and 
y. We use rem for the remainder function; rem(a, b) is the remainder 
when a is divided by b. 

if rem(y, x) — 
then output := x 

else do in parallel x :=rem(y, x) , y := x enddo 
endif 

Here is a variant, using subtraction instead of division. In fact, this 
variant corresponds more closely to what Euclid actually wrote; see 
[IH Propositions VII.2 and X.3]. 
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if x < y then y := y — x 
elseif y < x then x := x — y 
else output:= x 
endif 

Notice that the second version in effect uses repeated subtraction 
(rather than division) to compute in several steps the remainder func- 
tion rem used in a single step of the first version. 

Are these two versions the same algorithm? As we shall discuss 
later, this issue is debatable. The point we wish to make here, before 
entering any such debate, is that the use of low-level programs like 
Turing machines might cut off the debate prematurely by making the 
two versions of Euclid's algorithm identical. Consider programming 
these versions of Euclid's algorithm to run on a Turing machine, the 
input and output being in unary notation. The most natural way 
to implement division in this context would be repeated subtraction. 
With this implementation of division, any difference between the two 
versions disappears. 

If we were to use Turing machines (or similar low-level programs) 
as our standard, then to avoid letting them preempt any discussion of 
whether the two versions of Euclid's algorithm are the same, we would 
have to implement at least one of the two versions in a special way. 
For example, we might implement division by converting the inputs 
to decimal notation and having the Turing machine perform division 
as we learned to do it in elementary school. But then the question 
arises whether the preprocessing from unary to decimal notation has 
altered the algorithm. And once preprocessing enters the picture, there 
are arbitrary choices to be made (decimal or binary?), which further 
complicate the picture. 

Motivated by examples like the preceding one, we refuse to restrict 
ourselves to low-level programming languages. We want a language 
(or several languages) capable of expressing algorithms at their natural 
level of abstraction, not forcing the programmer to decide implementa- 
tion details. This desideratum, combined with the earlier one that the 
language (s) should have a precise, unambiguous semantics, severely 
restricts the supply of candidate languages. In fact, the only such 
language that we are aware of is the abstract state machine language, 
which, in our present context, means small-step (also called sequential) 
ASMs, as presented in pTj . 

Remark 2. To make this paper more self-contained, we give the follow- 
ing brief description of sequential ASMs. For more details, see [6] or 
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[TO] or [UJ . The rest of this paper does not depend on the exact defini- 
tion of ASMs; a reader willing to read ASM programs as pseudo-code 
should have no difficulty with their use here. 

An ASM program describes operations to be performed on a state. 
The state is a structure in the usual sense of first-order logic. The 
computation progresses in discrete steps, changing the interpretations 
of certain function symbols in the state. The basic units from which 
ASM programs are built are update rules of the form f(ti, . . . , t^) := t , 
whose meaning is that the value of (the interpretation of) the fc-ary 
function symbol / at the fc-tuple consisting of (the values of) t 1 , . . . , t fc 
is to be changed to (the value of) to- ASM programs are built from 
update rules by two constructors. One is parallel composition; 

do in parallel R\, . . . , R n enddo 

means to make all the state changes given by the rules Ri, . . . , R n . (If 
two of those changes clash, prescribing different updates of the same 
function at the same arguments, then no changes are to be made.) The 
other constructor is the conditional 

if (p then Rq else Ri, 

with the obvious meaning; here ip is a quantifier-free first-order formula. 
This concludes our rough description of ASM programs. 

We note here another advantage of ASMs: They can deal directly 
with structures, such as graphs, as input and output. This contrasts 
with the need, in most other programming languages, to code struc- 
tures. In the case of graphs, coding would involve not only deciding, for 
example, to represent graphs as adjacency matrices, but also choosing, 
for each particular input graph, an ordering of its vertices, because an 
adjacency matrix presupposes a particular ordering. 

ASMs also make it possible to write programs that use powerful, 
high-level operations. This is important for our purposes because we 
do not want the programming language to prejudge whether an al- 
gorithm using such operations is equivalent to one that implements 
those operations by means of lower- level ones. To avoid prejudging, 
the language must be able to formulate separately the high-level and 
low-level versions, not simply use the latter as a surrogate for the for- 
mer. The possibility of writing programs at different levels is one of 
the key strengths of ASMs. 

In view of these considerations, we shall use ASMs as our standard 
programs in this paper. We emphasize, however, that this decision 
is not essential for most of our arguments. The difficulties facing an 
attempt to define the "right" equivalence relation on programs are 
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intrinsic difficulties, not caused by our decision to use ASMs. In fact, 
the difficulties would be even worse if we were to use programming 
languages that lack a precise semantics or programming languages that 
cannot work at multiple levels of abstraction. 

3. Behavioral Equivalence 

A proper understanding of the use of ASMs to describe algorithms 
requires a look at the concept of behavioral equivalence, which plays a 
central role in the theory of ASMs. It is based on an abstract view of 
algorithms presented in Before discussing this point of view, it is 
necessary to resolve a terminological ambiguity, because the word "al- 
gorithm", which plays a crucial role in Shore's question, is also used as 
an abbreviation for "sequential algorithm" (as defined in [TT]) when no 
other sorts of algorithms are under consideration. We shall be careful 
to abstain from this abbreviation. "Sequential algorithm" will mean 
what is defined in namely an entity to which are associated a set 
of states, a set of initial states, and a transition function, subject to 
certain postulates]^ "algorithm" without "sequential" will mean what 
Shore asked for (and what we claim admits no precise characterization), 
essentially an equivalence class of programs with respect to the pie-in- 
the-sky equivalence relation. The terminology is somewhat unfortunate 
in that (1) it suggests that "sequential algorithms" are a special case 
of "algorithms", whereas the two concepts are actually separate and 
(2) the word "sequential" that is the difference between the two names 
is not a difference between the concepts, since we retain our earlier 
convention that the only algorithms (in Shore's sense) that we consider 
here are small-step, non-interactive ones. Nevertheless, it seems rea- 
sonable to use the expressions "sequential algorithm" and "algorithm" 
to match the terminology of the sources, [11] and [5] respectively. 

Behavioral equivalence, defined in [11] (but there called simply 
"equivalence") is a very fine equivalence relation on sequential algo- 
rithms, requiring that they have the same states, the same initial states, 
and the same transition function. The main theorem in [TT] is that ev- 
ery sequential algorithm is behaviorally equivalent to a (sequential) 
ASM. As was pointed out in [TT], behavioral equivalence seems exces- 
sively fine for many purposes, but this circumstance only makes the 
main theorem of [11] stronger. 



2 A rough summary of the postulates is that states are first-order structures, the 
transition function involves only a bounded part of the state, and everything is 
invariant under isomorphisms of structures. 
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For our present purposes, it is useful, though not absolutely nec- 
essary, to know that behavioral equivalence is fine enough, that is, 
that two behaviorally equivalent sequential algorithms are equivalent 
in Shore's sense. This knowledge is useful because it ensures that, by 
using ASMs as our standard programs, we have representatives for all 
of the algorithms in Shore's sense. If it were possible for behaviorally 
equivalent programs to be inequivalent in Shore's sense, then there 
might be algorithms that, although behaviorally equivalent to ASMs, 
are not Shore-equivalent to any ASMs, and such algorithms would be 
overlooked in our discussion. This would not be a disaster - if we con- 
vince the reader that there is no precise definition for Shore-equivalence 
on ASMs, then there is surely no such definition for an even wider class 
of programs (this is why we described the knowledge above as not abso- 
lutely necessary) — but it is desirable to know that we are dealing with 
the whole class of algorithms that we intended to deal with. 

Is this desirable knowledge in fact correct? Must behaviorally equiv- 
alent algorithms be equivalent in the sense that Shore asked for? In the 
rest of this section, we argue that the answer is yes, as long as we do not 
read into ASMs more information than they are intended to convey. 
In brief, the point is that ASMs are intended to describe sequential al- 
gorithms and, since these are not specified beyond their states, initial 
states, and a transition functionjl there is no room for any distinction 
between behaviorally equivalent ASMs or sequential algorithms. 

Because the ASM syntax is readable as pseudo-code, however, there 
is a temptation to consider an ASM as providing additional information 
beyond the sequential algorithm that it defines. Here is an example to 
illustrate the situation. 

Example 3. We describe here, in ASM form, a binary decision tree 
algorithm. The vocabulary has 0-ary relation symbols (also known as 
propositional symbols) U s for every sequence s G {0, l}- 5 (i.e., every 
sequence of at most 5 binary digits), it has constant symbols c s for 
s G {0, l} 6 , and it has one additional 0-ary symbol output. (Visualize 
the symbols U s as attached to the internal nodes s of a binary tree 
of height 7 and the symbols c s as attached to the leaves.) The only 
dynamic symbol is output. States are arbitrary structures for this 
vocabulary. The ASM program we want is best described by reverse 
induction on binary strings s of length < 6. If s has length 6, let Tl s 



3 The definition of "sequential algorithm" [IT] says that they are entities associ- 
ated with the three items mentioned, but the postulates concern only these three 
items. No further role is played by the entities themselves. 
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be the program 

output := c s . 

If s has length < 5, let U s be 

if U s then II S — (i) else n s — /q\. 

The program we are interested in is U where is the empty sequence. 
Its natural, intuitive interpretation IS CIS db computation that begins by 
looking up U ; depending on the answer, it looks up U s for s = (0) or 
s = (1); depending on the answer, it looks up U s for an appropriate s 
of length 2; and so forth, until, after 6 queries, it sets output to the 
appropriate c s . In other words, we have a standard binary decision 
tree. 

The algorithm does all this work in one step. (If it were allowed to 
run for another step, nothing would change in the state.) Notice that 
the program has length of the order of magnitude 2 6 and that its natural 
interpretation as a computational process looks up 7 values, namely 6 
Ug's and one c s . (To make precise sense of "order of magnitude" here 
and similar notions below, pretend that 6 is a variable.) 

Now let us apply the proof of the main theorem in [TTJ to the se- 
quential algorithm defined by this ASM. The proof produces another 
ASM, II', behaviorally equivalent to our IT . This ASM (see [TT] 
Lemma 6.11]) is a parallel combination of guarded updates, one for 
each of the 2 63 possible systems of values of the C/ S 's. So II' has length 
of the order of magnitude 2 2 , exponentially greater than U . Fur- 
thermore, in each run of the natural, intuitive interpretation of II' as a 
computational process, it looks up the values of all 63 of the t/ s 's. 

In view of the vastly greater work done by II' in each run (and its 
vastly greater length), it would be difficult to convince people that it 
represents the same algorithm as n . Yet, all the complexity of II' is 
hidden if one looks only at states and transition functions, and so IT' 
is behaviorally equivalent to IT . We assert that IT and U should be 
regarded as defining the same algorithm, that is, as equivalent programs 
in Shore's sense. The apparent differences between them arise from 
regarding the ASM programs not merely as descriptions of sequential 
algorithms but as telling in detail how the transition function is to be 
computed. It is tempting to assign this sort of detailed operational 
semantics to ASMs, but this conflicts with our intention to use them, 
as in (TTJ, simply to describe sequential algorithms. 

We do not claim that one should ignore the difference between an 
algorithm that looks up seven values and performs six tests on them 
and an algorithm that looks up 63 values and performs approximately 
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2 63 tests on them. But we do claim that, if these properties of the 
algorithms are to be taken into account, then the ASMs representing 
the algorithms should be designed to make these properties visible in 
the states and transition functions. That this can be done is part of 
the ASM thesis: Any algorithm can be represented at its natural level 
of abstraction by an ASM. In the case at hand, the level of abstraction 
could involve keeping track of which U s 's the algorithm has used. This 
can be done, for example, by having additional dynamic symbols V s 
(with value false or undef in initial states) with the intended meaning 
U U S has been used" and by including in both IT and II' updates setting 
V s to true whenever U s is use dfl Similarly, if we wish to take into 
account the huge number of guards (of conditional rules) evaluated in 
the computation intuitively described by IT, then we should write the 
ASM at that level of abstraction, that is, we should include updates 
whereby the algorithm keeps a record of all this work. 

For the rest of this paper, we shall take for granted, with the support 
of considerations like the preceding paragraph, that, whatever Shore's 
pie-in-the-sky equivalence relation ought to be, it will be at least as 
coarse as behavioral equivalence. 

Remark 4. Udi Boker [personal communication] has suggested that, 
once it is clear that ASMs are to serve as descriptions of what is to hap- 
pen as a result of a single step, not caring about how it is accomplished 
within the step, then behavioral equivalence captures the intuitive no- 
tion of algorithm. We feel that, although behavioral equivalence is a 
natural and important equivalence relation, it need not capture the 
intution. In Example [1] above and the related Example [5] below, we 
have algorithms that are not behaviorally equivalent but can neverthe- 
less be reasonably viewed as the same algorithm. In other words, the 
intuitive notion of "the same algorithm" seems not to require the com- 
putations to match step by step. Nor does it require the vocabulary 
of the states to be the same; a reasonable renaming of identifiers in 
a program should not change the algorithm (but see also Example [T51 
below). 

Given that Shore's pie-in-the-sky equivalence relation should be at 
least as coarse as behavioral equivalence, the main theorem of [TT] 



Another way to make the evaluation of U s 's visible for the purposes of behavioral 
equivalence is to make the C/ S 's external function symbols, so that evaluating them 
involves a query to the environment. In this paper, however, we prefer to consider 
only non-interactive algorithms. 
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assures us that ASMs are adequate for representing all sequential al- 
gorithms in the sense of [TTJ. These are exactly the small-step, non- 
interactive algorithms that we intend to treat in this paper. We there- 
fore have an adequate set of programs, the ASMs, with a precisely 
defined semantics. That is, we have a set of programs on which it 
makes sense to try to define an equivalence relation of the sort Shore 
asked for. Now that this prerequisite for Shore's question is satisfied, 
we turn to the question itself. 

4. Subjectivity 

The formulation of Shore's question, to "capture the notion that 
two algorithms are the same," presupposes that there is such a notion. 
And indeed, people do seem to have such a notion, to use it, and to 
understand each other when they use it. But it is not clear that they 
all use it the same way. Will two competent computer scientists (or 
mathematicians or programmers) always agree whether two algorithms 
are the same? We have already quoted Yanofsky's doubt about this 
point: equivalence of algorithms is a subjective notion. 

There are situations where disagreement is almost guaranteed. Sup- 
pose X has invented an algorithm and Y later invents a somewhat 
modified version of it. X will be more likely than Y to say that the 
algorithms are really the same. 

Even when ulterior motives are not involved, there can easily be 
disagreements about whether two algorithms are the same. Consider, 
for example, the two versions of the Euclidean algorithm in Example (TJ 
one using division and the other using repeated subtraction. Are they 
the same? 

One can argue that they are different. The version with subtraction 
is slower. It can be implemented on simpler processors, for instance, 
children who have learned how to subtract but not how to divide. It 
is not behaviorally equivalent to the division version because it takes 
several steps to do what division did in one step. 

Despite all these differences, it seems that most mathematicians (and 
computer scientists?) would, when confronted with either of these algo- 
rithms, call it the Euclidean algorithm. Furthermore, if one just asks, 
'What is the Euclidean algorithm?" , the answer is usually the division 
version (see for example [U Section 8.8] or [7]), even though Euclid 
gave the subtraction version. To call the division version "Euclidean" 
strongly suggests that it is considered "the same" as what Euclid did. 

The subjectivity of the situation is further emphasized by the fol- 
lowing variant. 
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Example 5. For positive integers a and b, let app(a, b) be the multiple 
of b that is closest to a (possibly below a and possibly above, and in 
case of a tie then below), and let 

rem' (a, b) = \a — app(a, b)\ 

be the distance from a to the nearest multiple of b. Now replace rem 
by rem' in the division version of the Euclidean algorithm. Does that 
change the algorithm? Notice that it may happen that, in runs of 
the rem and rem' versions of the algorithm on the same input, only 
the initial and final states coincide; all the intermediate states may be 
different (and there may be different numbers of intermediate states). 
Nevertheless, both versions are called the Euclidean algorithm. 

Finally, what about the Euclidean algorithm applied not to integers 
but to polynomials in one variable (over a field)? It proceeds just like 
the division version in Example HJ but it uses division of polynomials. 
When a and b 7^ are polynomials, there are unique polynomials q and 
r such that a = bq+r and r has strictly smaller degree than b does. (We 
regard the zero polynomial as having degree —00.) Let Rem(a, b) denote 
this r, and replace rem by Rem in the division version above to get the 
Euclidean algorithm for polynomials. Is that a different algorithm? 

Remark 6. There is a more abstract, non-algorithmic proof that every 
two positive integers have a g.c.d. It consists of showing that the 
smallest positive element z of the ideal {mx + ny : m,n G Z} serves as 
the g.c.d. The main point of the proof is to show that this z divides 
both x and y. That proof implicitly contains an algorithm for finding 
the g.c.d. z. As before, we take the initial state to have x and y, the 
two numbers whose g.c.d. is sought, but this time x and y are static; 
there is one dynamic function symbol z, initially equal to x. 

if rem(x, z) 7^ then z := rem(x, z) 
elseif rem(y, z) 7^ then z := rem(y, z) 
else output:= z 
endif 

It seems that this algorithm is really different from the Euclidean 
one; it uses a different idea. But we are not prepared to guarantee that 
no one will regard it as just another form of the Euclidean algorithm. 
Nor do we guarantee that "the same idea" has a clear meaning. 

Here are some more examples where it is debatable whether two 
algorithms should be considered the same. 
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Example 7. The first is actually a whole class of examples. Consider 
two algorithms that do the same calculations but then format the out- 
puts differently. Are they the same algorithm? Our description of the 
situation, "the same calculations" suggests an affirmative answer, but 
what if the formatting involves considerable work? 

Example 8. Is a sorting algorithm, like quicksort or mergesort a single 
algorithm, or is it a different algorithm for each domain and each linear 
ordering? This question is similar to the earlier one about the Euclidean 
algorithm for numbers and for polynomials, and it is clear that one 
could generate many more examples of the same underlying question. 

Notice that there are some situations that can be regarded as falling 
under both of the preceding two examples. Suppose one algorithm 
sorts an array of numbers into increasing order and a second algorithm 
sorts the same inputs into decreasing order. They can be viewed as the 
same algorithm with different formatting of the output, or they can be 
viewed as the same algorithm applied to two different orderings of the 
natural numbers. 

Example 9. Quicksort begins by choosing a pivot element from the 
array to be sorted. One might always choose the first element, but a 
better approach is to choose an element at random. A variant uses 
the median of three elements (for example, the first, middle, and last 
entries of the array) as the pivot. (See, for example, [TJ pages 94 and 
95].) Are these different algorithms? What if one increases "three" to 
a larger odd number? 

5. Purpose 

In the preceding section, we attributed variations in the notion of 
"the same algorithm" to people's different opinions and subjective judg- 
ments. There is, however, a more important source of variation, even 
between the notions used by the same person on different occasions, 
namely the intended purpose. 

For someone who will implement and run the algorithm, differences 
in running time are likely to be important. A difference in storage 
requirements may be very important on a small device but not on 
a device with plenty of space. Of course, the distinction between a 
small device and one with plenty of space depends on the nature of the 
computation; even a supercomputer is a small device when one wants to 
predict next week's weather. Thus, when one considers a specific device 
and asks whether an algorithm's space requirements are an essential 
characteristic of the algorithm, that is, whether one should count two 
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algorithms as different just because one uses much more space than the 
other, then answer is likely to be "yes" once the space requirements 
are large enough but "no" if they are small. Thus, two programs may 
well count as the same algorithm for someone programming full-size 
computers but not for someone programming cell-phones. 

Although we generally consider only non-interactive algorithms in 
this paper, let us digress for one paragraph to consider (a limited degree 
of) interaction. If an algorithm is to be used as a component in a 
larger system, additional aspects become important, which may be 
ignored in stand-alone algorithms. And in this respect, even just the 
interaction between an algorithm and the operating system can be 
important for some purposes. Two programs that do essentially the 
same internal work but ask for different allocations of memory space 
(or ask for the same allocation at different stages of the computation) 
or that make different use of library calls will be usefully considered 
as different algorithms for some purposes but probably not for all. 
Similarly, if an algorithm's output is to be used as the input of some 
other calculation, then formatting (see Example [7j) is more important 
than it otherwise would be. 

Returning to the standard situation of non-interactive, small-step 
algorithms, we present an example which, although it arose in a differ- 
ent context (complexity theory), helps to show how one's purpose can 
influence whether one considers two algorithms the same. 

Example 10. Suppose we have programs P and Q, computing functions 
/ and g, respectively, say from strings to strings, and each requiring 
only logarithmic space and linear time (in a reasonable model of compu- 
tation). Can we combine P and Q to compute the composite function 
fog in linear time and log space? The simplest way to compute fog on 
input x would be to first use Q on input x to compute g(x) and then 
use P on input g(x) to compute f(g(x)). This combined algorithm 
runs in linear time but not in log space; storing the intermediate result 
g(x) will in general require linear space. 

There is an alternative method to compute f(g(x)). Begin by run- 
ning P, intending to use g(x) as its input but not actually computing 
g(x) beforehand. Whenever P tries to read a character in the string 
g(x), run Q on input x to compute that character and provide it to P. 
But make no attempt to remember that computation of Q. When P 
needs another character from g(x), run Q again to provide it. Because 
this version of the algorithm does not try to remember g(x), it can run 
in log space. But because it needs to restart Q every time P wants 
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a character from g(x), it will run in general in quadratic rather than 
linear time. 

So we have two programs that combine P and Q to compute fog. 
One runs in linear time, and the other runs in log space, but neither 
manages to do both simultaneously. Do these two programs represent 
the same algorithm? 

In an application where strict complexity restrictions like linear time 
and log space are important, these programs should be considered dif- 
ferent. But in more relaxed situations, for example if quadratic time is 
fast enough, then the difference between these two programs might be 
regarded as a mere implementation detail. 

So far, the purposes we have discussed, which may influence the deci- 
sion whether two algorithms are the same, have involved implementing 
and running the algorithms. But people do other things with algo- 
rithms, besides implementing and running them. For someone who will 
extend the algorithm to new contexts, or for a mathematician who will 
appeal to the algorithm in a proof, the idea underlying the algorithm 
is of primary importance. Furthermore, in both of these situations the 
presuppositions of the algorithm will play an important role. Thus, for 
example, it would be an essential characteristic of (the division form 
of) the Euclidean algorithm that it uses a measure of "size" on the rel- 
evant domain (numerical value or absolute value in the case of integers, 
degree in the case of polynomials) such that one can always^ divide and 
obtain a remainder "smaller" than the divisor. 

It is reasonable to suppose that any worthwhile notion of sameness 
of algorithms will be adapted to some more or less specific purpose, 
and that different purposes will yield different notions of sameness. A 
global, all-purpose notion of "the same algorithm" is indeed pie in the 
sky. 

6. Equivalence 

Is the relation, between programs, of expressing the same algorithm 
really an equivalence relation? The preceding sections suggest that 
the relation depends on individual judgment and goals, but suppose 
we fix an individual and a goal, and use the resulting judgment; call 
two algorithms the same if and only if this particular person regards 
them as the same for this particular purpose. (And don't give him 
an opportunity to change his mind.) Now can we expect to have an 
equivalence relation? Shore's question explicitly asks for an equivalence 
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relation, but is the intuitive notion of "the same algorithm" necessarily 
an equivalence relation? 

6.1. Reflexivity. It seems obvious that every program defines the 
same algorithm as itself, that is, that no program defines two algo- 
rithms. But in fact, we had to take some precautions to ensure this 
obvious fact. Recall the example of the division form and the subtrac- 
tion form of the Euclidean algorithm which, when programmed on a 
Turing machine with unary notation, can become the same program. 
One of our reasons for not adopting Turing machines as our standard 
programs was precisely this possible failure of reflexivity. 

Our insistence that ASMs be understood as defining only sequential 
algorithms and in fact as defining only their states, initial states, and 
transition functions, not the details of what happens within a step, is 
also related to reflexivity. That is because the semantics of ASMs is 
defined only at the level of sequential algorithms. There is no guarantee 
that an ASM program cannot be regarded, by two readers, as describing 
two different processes within a step; the only guarantee is that the 
ultimate outcome of the step must be the same. 

We believe that we have taken enough precautions to ensure reflex- 
ivity. In fact, our discussion of behavioral equivalence in Section [3] was 
intended to support the thesis that an ASM program defines the same 
algorithm as any behaviorally equivalent ASM, in particular itself. 

6.2. Symmetry. The colloquial usage of "the same algorithm" seems 
to admit some mild failures of symmetry. For example, given a program 
A at a rather abstract level and a detailed implementation B of it, 
a person reading the text of B and suddenly understanding what is 
really going on amid the details might express his understanding by 
exclaiming "Oh, B is just the same as A" , while the symmetric claim 
"A is just the same as B" is unlikely. 

Nevertheless, the intuitive notion of "the same algorithm" underlying 
Shore's question is clearly a symmetric one. Shore's description of the 
intended intuition refers to the unordered pair of programs, using 
the phrases "two particular programs" and "two algorithms" with no 
distinction between a first and a second. 

Furthermore, someone exclaiming, in the situation described above, 
U B is just the same as A" would probably, if pressed, concede that he 
didn't really mean that they are "the same" but that B is merely an 
implementation of A, i.e., that the idea behind B is the same as that 
behind A, but B contains more details and thereby hides the idea. 
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6.3. Transitivity. In view of the preceding brief discussion of reflexiv- 
ity and symmetry, we regard the question "Do we have an equivalence 
relation?" as coming down to "Do we have transitivity?", and here 
things are considerably less clear. Might there be, for example, finite 
(but long) sequences of programs in which each program is essentially 
the same as its neighbors (and so should express the same algorithm), 
yet the first and last programs would not be considered the same al- 
gorithm? Here are some examples to indicate the sorts of things that 
can happen. The first example is a continuation of the binary decision 
tree example. 

Example 11. Consider the programs IT and IT described in Example [3j 
and modify them, as described earlier, to make explicit the differences 
in the work they do within a step. For example, have them update 
Boolean variables V s with the meaning U U S was used." These modifi- 
cations, call them Yl and IT, can reasonably be regarded as different 
algorithms. 

There is, however a "continuous" (in a discrete sense) transition be- 
tween them. An intermediate stage of this transition would be an 
algorithm that begins by looking up the values of U s for the first k se- 
quences s (in order of increasing length and lexicographically for equal 
lengths, i.e., for the first few levels of the decision tree and for some ini- 
tial segment of the next level). Using the answers, the algorithm finds 
(by a single parallel block of guarded updates, i.e., by a table look-up in 
a table of size 2 fc ) the path through the part of the tree about which it 
asked. It reaches a node s for which it knows the value of U s but didn't 
look up Ut for the children t of s. The value of U s tells the algorithm 
which of the two children t of s is relevant, so it looks up the value of 
that U t . This value tells which child of t is relevant, so that determines 
the next U to evaluate, and so forth. (The algorithm also updates the 
V^'s as above, to indicate which f/'s it evaluated.) When k = 1, this 
algorithm is TL . When k = 63, it is II'. These look different. But is 
there a significant difference between the algorithms obtained for two 
consecutive values of fc? It looks like just a minor bookkeeping dif- 
ference. So we have a sorites situation, going from one algorithm to 
an arguably different one in imperceptible steps — imperceptible in the 
sense that one could reasonably consider the k and k + 1 versions mere 
variant implementations of the same algorithm. 

Example 12. Similarly, in Example the quicksort algorithm could 
reasonably be considered unchanged by a minor change in the number 
k of elements whose median determines the first pivot. But a succession 
of such minor changes can lead to a situation where k becomes as large 
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as the entire array to be sorted. Then finding the median, to serve as 
the first pivot, is practically as hard as sorting the whole array. So at 
this stage, the algorithm is much less likely to be regarded as the same 
as traditional quicksort. 

The next example involves what may be, at least for logicians, the 
archetypal example of a trivial syntactic change that makes no real dif- 
ference, namely renaming bound variables (also called a-equivalence) . 
The particular choice of bound variables is so unimportant that they 
are often eliminated altogether, for example by means of de Bruijn in- 
dices in the A-calculus [I] or by the boxes-and-links scheme of Bourbaki 

i- 

Example 13. Sequential non-interactive ASMs, which we have chosen 
as our standard programs, don't have bound variables. They can, how- 
ever, and often do have variables that resemble bound variables in that 
the algorithm shouldn't care what those variables are (as long as they 
are distinct and don't clash with other variables). For example, many 
sequential ASMs have a dynamic symbol mode, serving as a sort of pro- 
gram counter, whose value in every initial state is a particular element 
(named by) start and whose value at the end of every computation is 
final. Neither the name mode nor the names of its intermediate values 
make any real difference to the computation. Nevertheless, changing 
these names produces a different ASM, one not behaviorally equivalent 
to the original because it has a different set of states (for a different 
vocabulary). At first sight, it seems clear that two programs that differ 
only by such a renaming should be equivalent; the underlying algorithm 
is the same. 

But for certain purposes, there may be a real difference, especially if 
the identifiers become very long strings, for example the entire text of 
War and Peace. Then an actual implementation will involve reading 
the identifiers and (at least) checking which of them are the same and 
which are not. If the identifiers get too long, a real- world interpreter or 
compiler will crash. (For such reasons, compiler expert Gerhard Goos 
likes to say that every compiler is almost always wrong, meaning that 
it is correct on only finitely many programs.) Even if the system (in- 
terpreter or compiler) is willing to handle very long identifiers, it must 
do additional work to distinguish them. Suppose, for example, that we 
take a program written with normal-sized identifiers and modify it by 
attaching a particular long random string to the beginning of all the 
identifiers. Then, if the compiler reads from left to right, it will have 
to read and compare long strings just to tell whether two occurrences 
are of the same identifier. (We could also attach another long random 
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string, of possibly different length than the first, to the end of each 
identifier, so as to defeat compilers that don't just read from left to 
right.) 

In accordance with the discussion in Section [31 if we want to take 
into account the work involved in reading and comparing identifiers, 
then we should write the ASM in a way that makes this work explicit. 
Suppose then that this rewriting has been done. Now the example leads 
to a sorites paradox. Lengthen the identifiers gradually; at what point 
do you get a different algorithm? For a fixed compiler (or interpreter), 
there may be a well-defined boundary, but the boundary moves when 
the compiler changes. 

Finally, here is an example arising from how the first author, after 
grading exam papers, sorts them into alphabetical order for recording 
the grades and returning the papers to the class; see also [T5j . 

Example 14. With N exams, first find the power of 2, say 2 k , such 
that s = N/(2 k ) is at least 3 but less than 6. (We don't have classes 
with fewer than 3 students, so k is a non-negative integer.) Partition 
the given, randomly ordered stack of exams into 2 k substacks of size s 
(rounded up to an integer in some stacks and down in others), and sort 
each substack by insertion sort (i.e., by inspection, since the stacks are 
so small). Merge these stacks in pairs to produce 2 k ~ l larger, sorted 
stacks. Then merge these in pairs, and so forth, until all the papers 
are in one sorted stack. (Implementation detail: If desk space is lim- 
ited, then don't do all the insertion sorts first and then all the merging. 
Rather, the stacks that would eventually be merged by the algorithm 
above should be actually merged as soon as they become available. 
This keeps the number of sorted stacks bounded by k instead of by 2 k . 
Although it's irrelevant to the purpose of this example, the variation 
raises again the question whether it changes the algorithm.) The deci- 
sion to have s between 3 and 6 is just a matter of convenience, so that 
insertion sort can be easily applied to the stacks of size s. In princi- 
ple, the bounds could be any s and 2s. For a fixed N, varying s gives 
a chain of algorithms connecting insertion sort (when 2s > N) and 
mergesort (when s = 1), such that each two consecutive algorithms in 
the chain are intuitively, at least for some people's intuition, not really 
different algorithms. 

If N is not fixed, then there is an infinite sequence of algorithms as 
above, indexed by the natural numbers s. But the sequence, which 
starts at mergesort, doesn't reach insertion sort. Instead, there is an- 
other sequence, indexed by increasing k (in the notation above), that 
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starts at insertion sort (when k = 0) and heads toward mergesort. Be- 
tween the two sequences, there are other algorithms (or other versions 
of the same algorithm), for example one that adjusts k (and therefore 
s) to do half of the work by insertion and half by merging. (It may 
be useful to think of this chain of algorithms in a non-standard model 
of arithmetic, where N can be infinite, and we have an internally fi- 
nite but really infinite chain of algorithms, indexed by k, connecting 
insertion sort to mergesort.) 

Example 15. Similar comments apply to the usual implementation of 
quicksort, where, as described in [T3(, page 116], "Subfiles of M or fewer 
elements are left unsorted until the very end of the procedure; then a 
single pass of straight insertion is used to produce the final ordering. 
Here M > 1 is a parameter . . . ." A small change in the parameter M 
can be viewed as a mere implementation detail, not a change in the 
algorithm, but a succession of such changes can lead, for files of a fixed 
size, from quicksort to insertion sort. The situation is just like that for 
mergesort except that insertion sort is used at the end of quicksort and 
at the beginning of mergesort. 

In all the preceding examples, we had a finite sequence of programs 
such that a reasonable person might call each particular consecutive 
pair of algorithms equivalent but might not want to call the first and 
last programs of the sequence equivalent. There is no clear transition 
from equivalent to inequivalent, just as in the classical sorites paradox 
there is no clear transition from a pile of sand to a non-pile (or from 
a non-bald head to a bald one). That situation casts doubt on the 
transitivity of the supposed equivalence relation. 

Vopenka and his co-workers have developed an interesting approach 
to such situations, using the so-called alternative set theory; see [20] , es- 
pecially Chapter III. This theory distinguishes sets from classes, but the 
distinction is not just a matter of size as in the traditonal von Neumann- 
Bernays-Godel or Kelley- Morse theories of sets and classes. Rather, 
classes are to be viewed as having somewhat more vague membership 
conditions than sets do. As a result, a subclass of a set need not be a 
set. 

The axiomatic basis of this theory allows a distinction between nat- 
ural numbers (in the usual set-theoretic sense) and finite natural num- 
bers. If a set contains and is closed under successor, then it contains 
all the natural numbers, but a class with the same properties might 
contain only the finite ones. The latter can be regarded as formally 
modeling the notion of a feasible natural number. If an equivalence 
relation is a set (of ordered pairs), then its transitivity property can 
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be iterated any natural number number of times, but if it is a class 
of ordered pairs, then only a feasible number of iterations will work. 
Thus, sorites paradoxes disappear if one accepts that feasibly many of 
grains of sand do not constitute a pile and that a man with feasibly 
many hairs is bald. These ideas can be similarly applied to our exam- 
ples above. (We thereby add credence to the first author's claim that 
the number of exams he has to grade is not feasible.) 

7. Analogies 

The issues involved in defining equivalence of algorithms are similar 
to issues arising in some other contexts, and we briefly discuss a few of 
those here. 

7.1. Clustering. The basic problem of saying when two things (in our 
case, programs) should be treated as the same is very similar to the 
basic problem of clustering theory, namely saying which elements of a 
given set are naturally grouped together. Indeed, if we had a reasonable 
metric on the set of programs, then we might consider applying the 
techniques of clustering theory to it. But before doing so, we should 
take into account a comment of Papadimitriou |17j : 

There are far too many criteria for the 'goodness' of a 
clustering . . . and far too little guidance about choos- 
ing among them. . . . The criterion for choosing the 
best clustering scheme cannot be determined unless the 
decision-making framework that drives it is made ex- 
plicit. 

And that brings us back to our earlier comment that the notion of 
"same algorithm" that one wants to use will depend on what one in- 
tends to do with it and with the algorithms. 

We should also note that Papadimitriou's discussion is in the context 
where the metric is given and the issue is to choose a criterion for 
goodness of clustering. Our situation is worse, in that we do not have 
a natural metric on the set of programs. 

7.2. Proofs. Another analog of 'When are two algorithms the same?" 
is "When are two proofs the same?" The role played by the set of 
programs in our discussion of algorithms would be played by the set 
of formal deductions in some axiomatic system. The problem is to say 
when two formal deductions represent the same proof. There are many 
axiomatic systems that one might consider, ranging from propositional 
calculus up to systems like Zermelo-Fraenkel set theory that provide a 
foundation for almost all of mathematics. But the question of which 
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deductions represent the same proof makes sense at each level, and it 
leads to many of the same issues that we have discussed in connection 
with algorithms. 

Equality of proofs has been studied in considerable detail but usu- 
ally only at the level of elementary logic; see for example [H] and [9]. 
Renaming bound variables is regarded as not changing a proof; the 
same goes for cancelling the introduction of a connective or quantifier 
and an immediately following elimination of that connective or quanti- 
fier. (This formulation refers to the introduction and elimination rules 
of natural deduction systems, but there are analogous notions for se- 
quent calculi. Hilbert-style calculi are not used for studies of equality 
of proofs, because they require circumlocutions that make formal de- 
ductions bear little resemblance to the intuitive proofs they should ex- 
press.) But such equivalence relations between deductions are too fine 
to capture the intuitive notion of proofs being the same. In this respect, 
they resemble the equivalence relation described in [21]. In fact there 
is another similarity here, namely that in both cases the equivalence 
relations are designed to look nice from a category-theoretic viewpoint. 

Considerably larger rearrangements of the material in a deduction 
would be recognized by mathematicians as not changing the proof that 
the deduction represents. Other changes, however, are considered es- 
sential, for example the difference between an analytic and an elemen- 
tary proof in number theory, or the difference between a bijective proof 
and a manipulation of generating functions in combinatorics. Between 
the "clearly the same" and "clearly different" cases, there is a gray area 
that looks quite analogous to what we see in the case of algorithms. 

The analogy between proofs and algorithms has been given math- 
ematical content by the propositions-as-types or Curry-Howard cor- 
respondence. Here formal deductions in certain systems correspond 
exactly to programs in certain (usually functional) programming lan- 
guages. So there may be reason to think that, if we thoroughly un- 
derstood sameness in one of the two contexts, then we could transfer 
that understanding (or at least part of it) to the other. Unfortunately, 
the existing notions of equivalence of proofs translate into equivalence 
relations on algorithms that are too fine; they count algorithms as 
equivalent only if they are so for trivial reasons. 

In addition to the Curry-Howard correspondence, there is another 
connection between proofs and computation. If we take an algorithm 
for computing a function / and we run it on input x obtaining output 
y, then the record of the computation can be regarded as a proof, in 
an appropriate formal system, of the equation f(x) = y. (This is the 
essential point behind the theorem that all recursive functions on the 
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natural numbers are representable in (very weak) systems of arithmetic 
like Robinson's Q; for details see [121 Section 2.4] and note that what 
is nowadays called representable was there called definable.) If we had 
a good notion of equivalence of proofs in such formal systems, then we 
could use it to define a notion of equivalence of algorithms. Call two 
algorithms, for computing the same function /, equivalent if, for each 
input x, not only do they produce the same output y, but the resulting 
proofs of f(x) = y are equivalent. Unfortunately, it seems that, just 
as in the case of the Curry-Howard correspondence, existing notions 
of equivalence of proofs are too fine and therefore so are the resulting 
notions of equivalence of algorithms. 

7.3. Other analogous questions. We already mentioned, in Re- 
mark [6l that certain algorithms are considered different because they 
are based on different ideas. So we ask: What exactly does it mean for 
two ideas to be different? What is the "right" equivalence relation on 
ideas? This problem looks even worse than the original question about 
algorithms, partly because of its great vagueness and partly because 
the only analog for the set of programs, a set of things that can ex- 
press ideas, would seem to be something like the set of all (English?) 
texts, and the ways in which a text can express an idea seem to be 
entirely out of control. A related question about texts is 'What does it 
mean for a text in one language to be a translation of a text in another 
language?" Is expressing the same idea a necessary condition? Is it 
a sufficient condition? Does it depend on the chosen notion of "same 
idea'"? 

Moschovakis [H] has argued that the meaning of a term in English 
(or other natural languages) is the algorithm for computing its denota- 
tion. With this identification of meanings with algorithms, the question 
of when two ideas are the same would be not merely analogous to the 
question of when two algorithms are the same; the former would be- 
come a special case of the latter, at least for those ideas that occur as 
the meanings of terms. 

Similar issues arise in mathematics. What does it mean to say that 
two theorems are equivalent? Material equivalence, meaning equality 
of truth values, is certainly not the intention. The intuitive idea is 
that each of the theorems follows easily from the other - at least more 
easily than proving either theorem from scratch. But "easy" and "from 
scratch" are subjective notions. 

Reverse mathematics (see [H [18]) provides a notion of equivalence 
of theorems, namely provable equivalence over some weak base theory. 
This notion is, however, much coarser than the intuitive notion, since 
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the proofs of equivalence may be highly non-trivial, often more difficult 
(because of the weakness of the base theory) than the usual proofs 
(in strong theories like Zermelo-Fraenkel set theory) of the theorems 
themselves. Indeed, much of the fascination of reverse mathematics 
comes from the equivalence between theorems from very different fields 
of mathematics, theorems that at first seem to have nothing to do 
with each other. Another way to view the coarseness of the notion of 
equivalence given by reverse mathematics is that it looks only at one 
aspect of theorems, namely the set-existence assumptions that underlie 
them, whereas the intuitive notion of equivalence would look rather at 
the (intuitive) content of the theorems. 

We trust that the reader can extend this list of examples. 
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